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Abstract 

The  Army’s  Center  of  Battlefield  Excellence  in  Human-Centric  Command  &  Control  Decision 
Making  is  exploring  how  to  use  information  visualization  to  enable  collaborative  sensemaking. 
The  goal  is  to  provide  a  common  operating  picture  with  shared  situation  awareness  in  the  context 
of  dynamic  task  situations.  We  have  developed  a  Sensemaking  Support  System  (S3),  a 
prototype  sensemaking  visualization  tool  with  situation  understanding  capability  and 
knowledge  discovery  components.  We  experimentally  validate  the  utility  of  the  tool 
through  series  of  experiments  from  a  set  of  minimally  constructed  stories  (MCS)  that 
contain  the  saliencies  of  unstructured  battlefield  information  dynamics.  The  results  show 
the  followings:  (a)  the  perception  rating  of  S3  with  respect  to  sensemaking  cognitive 
measures  was  highly  significant;  b)  the  problem  scenarios  (MCS)  were  highly  significant; 
and  there  were  noticeable  interaction  effects  between  the  cognitive  measures  and  the 
problem  types.  Generally,  the  S3  software  needs  further  improvement  in  representation 
fidelity  of  problems  that  mimic  battle  field  situations.  This  is  shown  by  the  poor  weighted 
ratings  in  a  more  chaotic  scenario  MCS3. 

INTRODUCTION 


The  Backdrop 

Within  the  Department  of  Defense  (DOD),  team  decision  making,  shared 
situational  awareness  (SSA),  and  common  operating  picture  (COP)  have  become  the 
dominant  paradigms  or  lexicons  for  Command  and  Control  (C2)  decision  making 
process.  This  is  clearly  stated  in  the  Army’s  FM  3.0  doctrine  handbook.  The  FM  3.0, 
Operations,  refers  to  battle  space  visualization  in  several  ways.  Doctrinally,  these 
characteristics  generally  apply  to  visualization  in  any  type  of  operational  situation. 
Hence,  they  serve  to  frame  any  sort  of  analysis  that  might  be  undertaken  to  identify  key 
training  objectives  (Leedom,  2006).  As  noted  in  the  FM  3.0,  visualization  is  a  purposeful 
activity  since  it  enables  people  to  detect  the  elements  of  a  situation  before  making  any 
decision.  That  is,  one  engages  in  battle  space  visualization  for  the  specific  purpose  of 
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identifying  specific  actions  that  can  be  taken  to  influence  the  present  situation  and  move 
it  toward  an  intended  objective  or  end  state: 

...  Commanders,  assisted  by  the  staff,  visualize  the  operation,  describe  it  in  terms 
of  intent  and  guidance,  and  direct  the  actions  of  subordinates  within  their  intent... 
...  The  volume  of  available  information  challenges  all  leaders.  They  assimilate 
enormous  amounts  of  information  as  they  visualize  the  operation,  describe  their 
intent,  and  direct  their  subordinates  ’  actions.  Visualizing  the  operation  is 
continuous.  It  requires  commanders  to  understand  the  current  situation,  broadly 
define  the  future  situation,  assess  the  difference  between  the  two,  and  envision 
major  actions  that  link  them. 

The  Challenge 

With  major  combat  operations  in  both  Afghanistan  and  Iraq,  coalition  forces  face 
a  much  more  complex  challenge  in  the  furtherance  of  its  national  security  objectives  -the 
emergence  of  what  Colonel  T.X.  Hammes1  has  termed  fourth-generation  warfare.  One  of 
the  several  problems  in  this  kind  of  war  dimension  is  managing  information  equivocality 
and  responding  to  fast-space  changes  of  battle  space  dynamics.  This  leads  to  the 
requirement  of  changes  in  the  current  training  paradigms  to  include  sensemaking  in 
which  individuals  and  teams  of  soldiers  (platoons,  units,  etc)  are  taught  to  visualize 
changing  and  emerging  battle  field  dynamic  and  develop  situation  understanding  of  such 
situations.  A  common  operating  picture  (COP)  that  can  provide  dynamic  situation 
awareness  must  be  provided. 

Research  has  indicated  that  group  decision  making  participants  have  difficulty  in 
sharing  their  uniquely  held  information  or  tacit  knowledge  and  in  integrating  unique 
information  from  other  participants.  In  the  military  context,  or  even  in  the  business 
environment,  no  single  individual  can  claim  to  know  how  to  make  sense  of  information 
from  environments  that  change  over  time,  and  for  which  no  one  meaning  can  be  assigned 
to  a  contextual  piece  of  information.  In  a  typical  C2  environment,  shared  information 
items  might  include  rules  of  engagement,  order  of  battle,  Commander’s  Intent,  the 
OPLAN,  etc.  In  a  business  environment,  these  might  include  items  such  as  news  report 
by  a  television  network,  newspapers,  widely  circulated  company  documents,  ticker 
quotes,  etc.  The  battle  staff  working  in  groups  will  use  the  shared  information  in  context 
to  develop  a  sensemaking  process  so  as  to  derive  actionable  intelligence  or  a  situation 
understanding  necessary  to  inform  for  command  decision  making. 

SENSEMAKING 

Asymmetric  warfare  has  some  elements  of  complexity,  dynamism,  uncertainty, 
and  other  characteristics  that  defy  conventional  planning.  It  begs  for  deeper  cognitive 
analysis  that  is  grounded  on  both  ontological  and  epistemological  reasoning — seeking 
answers  to  “why”,  “when”,  “where”,  “how”,  so  on.  For  example,  in  the  September  11, 
1991  terrorist  attack  report,  so  many  questions  have  been  asked  —  What  happened? 
When?  Where?  Says  who?  Was  it  a  terrorist  act?  Did  it  involve  an  intelligent  failure? 
Trying  to  make  sense  of  the  situation  has,  and  will  continue  to  hunt  human  rationality. 
The  process  of  gaining  an  insight  into-  and  an  understanding  of-  a  situation  is  known  as 
making  sense.  When  the  understanding  is  translated  into  actionable  intelligence,  a 


1  COL  T.  X.  Hammes  (2004).  Hammes,  T.X.  (2004).  4th-generation  warfare.  Armed  Forces  Journal. 


situation  understanding  is  said  to  have  occurred  (Ntuen,  2006).  Sensemaking  is  then,  a 
process,  a  design,  or  a  technique  of  fusing  information  in  context  to  derive  understanding 
from  fragmentary  pieces  of  information. 

Sensemaking  can  be  viewed  as  a  paradigm,  a  tool,  a  process,  or  a  theory  of  how 
people  reduce  uncertainty  or  ambiguity;  socially  negotiate  meaning  during  decision 
making  events.  Weick  (1995)  states  that  sensemaking  refers  to  how  meaning  is 
constructed  at  both  the  individual  and  the  group  levels.  Through  the  accurate  construction 
of  meaning,  clarity  increases  and  confusion  decreases.  Leedom  (2002)  indicates  that 
battle  rhythms  can  best  be  understood  through  the  sensemaking  process.  A  poor 
sensemaking  process  often  leads  to  poorly  understood  objectives,  missions,  and  visions. 
This  in  turn  can  lead  to  poor  framing  of  plans,  and  consequently,  poor  decisions. 
Sensemaking  involves  the  collective  application  of  individual  “intuition” — experience- 
based,  sub-consciously  processed  judgment  and  imagination — to  identify  changes  in 
existing  patterns  or  the  emergence  of  new  patterns  (Weick,  1995).  A  peruse  of  literature 
on  sensemaking  can  be  summarized  as  follows:  How  are  meanings  and  understanding  of 
situations,  events,  objects  of  discourse,  or  contextual  information  produced  and 
represented  in  a  collective  context? 

THE  CURRENT  STATE-OF-THE  ART 

There  is  a  recent  interest  in  using  display  and  visualization  technology  to  enable 
the  sensemaking  process.  In  describing  the  overall  process  of  battle  command  as  an  art, 
Army  doctrine  infers  that  battle  space  visualization  is  frequently  an  intuitive  process  -that 
is,  one  governed  largely  by  automatic  cue  recognition  and  the  activation  of  tacit 
knowledge: 

...Using  judgment  acquired  from  experience,  training,  study,  and  creative 
thinking,  commanders  visualize  the  situation  and  make  decisions.  In  unclear 
situations,  informed  intuition  may  help  commanders  make  effective  decisions  by 
bridging  gaps  in  information.  Through  the  art  of  command,  commanders  apply 
their  values,  attributes,  skills,  and  actions  to  lead  and  motivate  their  soldiers  and 
units... 

Figure  1  below  portrays  this  doctrinal  representation. 

The  development  of  the  current  generation  Common  Operating  Picture  (COP) 
was  motivated  by  the  desire  to  improve  situation  awareness  within  a  military  command 
organization  -thus  leading  to  faster  and  better  synchronized  planning  and  execution 
decisions.  Like  many  information  management  systems  found  in  corporate  industry,  the 
COP  was  built  on  the  same  philosophy  used  for  managing  physical  assets:  capture 
information  and  put  it  in  a  place  where  it  can  easily  accessed.  The  next  generation  COP 
must  be  built  upon  a  clear  understanding  of  the  socio-cognitive  processes  employed 
within  the  military  command  organizations  to  translate  available  information  into  timely 
and  focused  action.  This  has  been  the  premise  of  sensemaking  support  systems;  much  of 
which  has  been  developed  implicitly  into  decision  support  tools. 

The  major  difference  between  sensemaking  support  system  (S3)  tools  and 
decision  support  systems  (DSS)  is  that  S3s  support  sensemaking  activities,  while  DSS 
support  decision  making  activities.  DSS  has  matured  in  its  constructs  and  theories;  S3s 
are  relatively  nascent  and  universally  lack  acceptable  theoretical  frameworks,  and 
constructs.  Nevertheless,  attempts  to  create  S3s  date  back  to  the  late  1980s  (“NoteCards” 


by  Halasz,  Moran,  &  Trigg;  “gIBIS”  by  Conklin  &  Begeman)  in  the  areas  of  Human 
Computer  Interaction  and  Computer  Supported  Cooperative  Work.  Other  influential  early 
systems  are  SEPIA  (Streitz,  Hanneman,  &  Thuring,  1989)  and  SIBYL  (Lee,  1990).  The 
systems  previously  mentioned  are  all  based  on  graphical  hypertext  and  were  definitely  a 
force  of  movement  towards  what  we  called  S3s.  Another  influential  tool  was  QuestMap, 
developed  in  1990.  It  was  created  to  support  sensemaking  and  group  activities.  QuestMap 
enjoyed  success  as  a  commercial  tool,  but  the  success  was  limited  and  the  system  did  not 


LEAD 


■  Pimislpl®®  □  f  wgr 

-’Tmrmrn 
Experience 


™r  r*  me 

■  Co 


METT-TC 

Common  operational  picture 


Visualize 

Art  of  War/ Science 

Mature  and  Decign 
of  Ilia  Operation 


Describe 

Ait  of  WaffSoianoa 

Time,  Space, 
RoMurcesr 
Purpose,  &  Action 


Shaping  opomuana 
Sustaining  operations 


Direct 

Art  of  Wa  r/Scisn 


Battlefield 

OparatJng 

Systems 


/ 

1\ 

1st  mates:  | 

£ 


*  I  ntsl^flfiqe 
<  Maneuver 

*  Fire  support 
-  Air  dtf^nEa 

*css 


-  End  state  and  military  oondltlonB 

■  Decltfve  point*  and  ebjectvoa 

*  Linns  o t  operation 

*  Culminating  paint 

■  Operational  reach,  approach,  and  pauses 

-  SI  muttaneo  ja  end  aaquentfcil  opomriona 

-  UniEarand  nonlinear  operations 

*  Tempo 


*  Plans  and  orders 

*  Pragfcsn&fion 

*  Execution 


-ASSESS 


Ligure  1.  Army’s  Doctrinal  &  Operational  Support  (LM  3.0,  pp.  5-4) 

stay  in  the  market  too  long.  Taking  advantage  of  the  lessons  learned  through  QuestMap, 
the  Compendium  (Selvin  et  al.,  2001)  methodology  was  initially  developed  by  Verizon 
research  labs,  and  it  was  first  released  as  a  computer  technology  in  2003.  Compendium 
seeks  to  improve  the  knowledge  management  and  sensemaking  ability  of  groups  and 
individuals.  These  systems  are  representative  of  some  of  the  most  influential 
developments  in  the  area  of  S3.  Table  1  shows  some  selected  S3  related  tools  over  the 
past  years.  It  should  be  noted  that  most  of  the  systems  listed  above  are  purely  theoretical, 
and  addressed  well  designed  problems  in  context.  The  last  five  rows  of  the  table  only 


provide  paradigms  for  design.  Therefore,  we  need  a  support  system  that  can  handle  the 
characteristics  of  modem  battlefield  information  management. 


Table  1.  Some  selected  related  S3  software  tool  and  paradigms 


Name/Year 

Developer/s 

NoteCards  (1987) 

Halasz  F,  Moran  T,  Trigg  R 

gIBIS  (1987) 

Conklin,  J.,  and  Begeman,  M.L. 

Working  by  Wire  SM  (1996) 

Gundry  &  Metes 

Sensemaker  (1997) 

Baldonado  and  Winograd 

KnowledgeX  -  KORE  (1998) 

IBM 

Vxlnsight  (1998) 

Davidson,  Hendrickson,  Johnson,  Meyers,  &  Wylie 

KIE  Sensemaker  Tool  (1998) 

Bell,  P 

PowerBookmarks  (1999) 

Li,  W.,  Vu  Q.,  Agrawal  ,D.,  Hara,  Y.,  Takano,H. 

Raven  (1994) 

Finkelstein  et  al. 

ClaimSpotter  (2004) 

Sereno,  Shum,  &  Motta 

Mission  Mate  (2004) 

Cheah,  Chew,  and  Tan 

eKnowledge  Suite  (2004) 

NewHyperG 

Confrontation  Manager  (2005) 

Crannell,  Howard,  Norwood,  Tait 

Joint  Battle  space  Infosphere  (2005) 

Milligan  and  Ahmed 

ClaiMaker,  ClaiMapper,  &  ClaimFinder  (2005) 

Uren,  Buckingham  Shum,  Bachler,  &  Li 

KnowledgeMiner  (2006) 

Knowledge  Miner 

VantagePoint 

Search  Technology 

CommonKADS  Methodology  (1999) 

Schreiber  et  al. 

Compendium  (2001) 

Selvin  et  al. 

Cynefin  Framework  (2003) 

Kurtz  &  Snowden 

SSIGS:  Sensemaking-supporting  Information 

Qu,  Yan 

Data/Frame  Model  (2004) 

Klein  et  al. 

Issue-Hypothesis-Evidence  framework  (2004) 

Leedom,  D.K 

Three-part  Workspace  Awareness  framework 

Gutwin  and  Greenberg 

COLAB  (2005) 

Morrison  and  Cohen 

SENSEMAKING  SUPPORT  SYSTEM  (S3)  DESIGN 
Foundation  Cognitive  Theory 

According  to  the  Army’s  FM  3.0,  “Commander’s  visualization  is  the  mental 
process  of  achieving  a  clear  understanding  of  the  force ’s  current  state  with  relation  to 
the  enemy  and  environment.  ”  This  alludes  to  the  fact  that  sensemaking  is  a  cognitive  task 
(Ntuen,  2006).  To  design  S3,  we  capture  this  doctrinal  statement  in  terms  of  cognitive 
abstraction  hierarchy  that  consists  of  four  macro  level  cognitive  elements.  These  are:  1) 
the  operator’s  experience  as  retrospective  knowledge,  2)  a  situation  awareness  model  that 
enables  spatial  knowledge  and  affordance  for  noticing  information,  3)  a  knowledge 


discovery  model  build  around  a  network  of  human  visualization  and  pattern  recognition 
mental  models,  and  4)  a  situation  understanding  model  to  enable  the  process  of 
translating  information  in  context  to  actionable  intelligence.  This  cognitive  hierarchy  is 
portrayed  in  Figure  2. 

Supporting  sensemaking  tasks  requires  that  we  consider  both  team  (shared)  and 
individual  situation  awareness  and  critical  thinking  skills.  As  shown  in  Figure  2,  a  critical 
thinking  skill  set  consists  of  the  requirements  needed  to  frame  an  understanding  of  a 
situation  from  the  stand  point  of  contextually  changing  goals  with  reference  to  the 
knowledge  elements  shown  on  the  right  hand  side  boxes  of  Figure  2.  It  also  includes  the 
ability  to  use  retrospective  knowledge  to  discover  new  knowledge  elements  from 
evolving  situations;  and  making  meanings  out  of  a  situation  in  order  to  supports  dynamic 
decision  making.  In  the  battlefield  C2  situations  such  decisions  must  be  cognizant  of  the 
distributed  information  networks  which  are  used  to  create  common  operational  pictures 
(COPs)  of  the  battle  space.  A  COP  facilitates  collaborative  planning  and  assists  all 
command  echelons  in  achieving  consistent  situation  awareness,  both  notionally  and 
practically  in  a  parallel  and  converging  manner.  In  this  context,  the  term  “picture”  refers 
not  so  much  to  a  graphical  representation,  but  rather  the  knowledge  used  to  define  an 
operational  situation. 


Situation 


Figure  2.  Cognitive  information  hierarch  in  S3 

THE  SENSEMAKING  VISUALIZATION  MODEL 

According  to  Leedom  (2006),  visualization  is  structurally  framed  by  doctrine  and 
intuition  (i.e.,  tacit  knowledge)  plays  an  important  role  in  framing  the  battle  space 
visualization  process.  These  visualization  models  should  be  able  to  capture  the 
commanders’  experience  as  well  as  the  doctrinal  guidance.  S3  attempts  to  capture  these 
knowledge  elements  in  three  ways:  (a)  The  Commanders’  perspective  and  the  things  they 
emphasize  when  trying  to  understand  a  situation;  (b)  The  elements  or  features  of 
battlefield  knowledge  as  visualized  by  the  commanders  and  the  battle  staff.  This  could 
take  various  forms  that  include  the  explication  of  commander’s  intent,  Commander’s 
critical  information  requirements  (CCIRs),  and  priority  intelligence  requirement  (PIR); 
and  (c)  the  operational  resource  requirements,  time,  purpose,  and  perceived  actions  as 
defined  by  such  doctrinal  elements  as  METT-TC  (Mission,  Enemy,  Time,  Terrain; 
Tactics  and  Civil  affairs).  These  requirements  may  differ  from  contexts  to  contexts  and  in 
operational  echelons;  e.g.,  a  context  may  be  urban  warfare  in  which  pursuit  tracking 
display  elements  are  used  to  identify  the  adversary,  a  stability  operation  in  which 
behaviors  of  local  insurgents  are  to  be  monitored;  at  the  tactical  level,  display  and 
visualization  elements  may  be  designed  to  support  a  real-time  understanding  of  the 
battlefield  dynamics  and  the  requirements  to  map  actionable  intelligence  to  bring  the 
desired  effects;  at  the  operational  level,  such  visualization  elements  may  be  designed  to 
allow  the  commanders  to  conduct  “what  if’  and  “what  next”  simulation  exercises  to 
assess  task  and  resource  requirements,  and  the  levels  of  effects  likely  to  result  from  the 
simulation. 

S3  is  aimed  primarily  at  the  team  sensemaking  exercises.  The  S3  design  is  to 
enable  information  sharing  in  a  community  of  battle  staffs.  As  noted  by  Eppler  and 
Burkhard  (2004),  knowledge  visualization  allows  visual  representations  to  improve  the 
creation  and  transfer  of  knowledge  between  people  by  sharing  what  they  know  and  what 
they  need  to  know  through  perspective  making  and  sharing.  This  collaborative 
knowledge  sharing  is  crucial  to  the  battle  staff  that  must  collectively  connect  their 
experiences  and  share  their  perspectives  on  how  the  battlefield  information  is  related  to 
the  mission  and  the  commander’s  intent.  Visualization  is  then  used  to  encourage  these 
shared  beliefs,  values,  experiences,  individual  intentions,  and  the  meanings  that  each 
battle  staff  gives  to  context  information.  On  this  note,  visualization  can  be  constructed  as 
a  social  interaction  model  in  which  group  cultural  cognition  can  emerge.  As  a 
community,  the  battle  staff  can  use  visualization  aids  to  create  and  reproduce  knowledge 
through  social  relationships  and  interactions  defined  by  common  standard  operating 
pictures,  doctrines,  and  tactics,  techniques,  and  procedures  (TTP).  However,  notes  Novak 
and  Wurst  (2004),  “in  order  to  make  sense  out  of  information  and  construct  knowledge, 
one  need  to  contextualize  it  within  one ’s  own  existing  knowledge  and  thought  world.  ” 

Our  S3  architecture  is  constructed  around  the  above  mentioned  collaborative 
cognitive  requirements  of  the  commanders  and  the  battle  staffs.  The  visualization  model 
is  driven  by  task  contexts  defined  at  different  strata  of  operational  doctrines.  Figure  3 


illustrates  the  theoretical  hierarchy  of  information  flow  and  their  model  elements.  As 
shown  in  Figure  3,  S3  uses  situation  stories  to  trigger  the  sensemaking  process.  We  use 
minimally  constructed  story  (MCS)  for  each  exercise  because  we  want  to  represent 


Figure  3.  The  information  model  components  of  S3 


different  levels  of  complexity  to  simulate  battlefield  dynamics  and  their  evolving 
information  contexts.  We  use  the  Cynefin  model  of  complexity  (Kurtz  &  Snowden,  2003) 
as  a  paradigm  for  this  purpose.  Like  the  Cynefin  model,  S3  structures  problems  into  four 
quadrants:  (1)  ordered  domain  with  known  causes  and  effects,  (2)  ordered  domain  with 
knowable  causes  and  effects,  (3)  un-ordered  domain  with  complex  relationships,  and  (4) 
un-ordered  domain  with  chaotic  situations.  Exhibit  1  shows  examples  of  MCS  used  in  S3. 
As  shown  in  Exhibit  1,  the  cases  are  minimal  in  terms  of  the  description  (completely 
open-ended)  and  vagueness  (no  specificity).  The  battle  staffs  and  the  intelligent  analysts 
usually  encounter  these  kinds  of  situations.  In  reality,  the  stories  may  be  more  detailed 
and  comprehensive,  and  can  be  obtained  from  the  descriptions  of  the  commander’s  intent 
or  operation  plan.  In  any  case,  the  sensemaking  process  requires  a  collaboration  between 
the  battle  staffs  as  each  individual  knowledge  are  processed  to  obtain  a  global  situation 
understanding  of  the  courses  of  actions  and  their  execution  orders. 


MCS  1:  A  battalion  XO  in  his  TOC  just  received  an  intelligence  report  on 
possible  attacks  on  critical  infrastructures  including  kidnapping  key  political  and 
religious  leaders  in  Basra,  Iraq. 

MCS  2:  A  group  of  terrorists  has  been  arrested  in  node  K  of  the  battle  network. 
Intel  shows  that  they  have  spread  to  other  cells  in  the  network,  but  are  not 
identifiable. 


Exhibit  1.  Sample  MCS  cases 


S3  Software  Tool 


Objective:  The  purpose  of  S3  is  to  create  display  and  visualization  capabilities  for  use  in 
collaborative  sensemaking  process.  The  intent  is  to  allow  a  team  of  sensemakers  to  (1) 
share  their  tacit  knowledge;  (2)  perform  critical  thinking  on  unstructured  problems;  (3) 
have  a  common  operating  picture  or  situation  awareness  of  the  problem  situation;  (4) 
allow  teams  to  propose  and  negotiate  solutions  through  a  convergence  of  shared  mental 
models;  and  (5)  allow  teams  to  visualize  alternatives  perspectives  from  other  team 
members  so  as  to  reach  a  common  understanding  of  the  battle  situation.  These  objectives 
support  many  of  the  core  C2  functions  stated  by  Alberts  &  Hayes  (2006)2. 

The  current  features  in  S3  allows  up  to  three  users  to  conduct  sensemaking  based 
on  an  MCS  (See  left  window,  upper  hand  comer  of  Exhibit  2).  Each  user  logs  into  S3 
with  a  protected  password  and  first  completes  team  dynamic  survey  (TDS).  The  right  side 
of  Exhibit  2  allows  the  team  leader  to  show  maps,  and  by  using  a  white  board  capability, 
can  illustrate  possible  locations  of  the  enemies  (red  bar),  possible  plans  of  engagement 
(blue),  and  avenues  of  approach.  S3  also  allows  the  user  to  search  for  information  using 
text  browsing  on  the  web.  Other  decision  information  can  be  displayed  through  use  of 
statistical  analysis  tools  on  Excel  spreadsheet  (See  Exhibit  3).  For  example,  the  user  may 
want  to  look  for  doctrinal  information  on  the  Army’s  web  page  or  CNN  news  report  on  a 
terrorist  bombing  in  Iraq.  S3  has  both  text  and  voice  capability.  The  user  may  decide  to 
turn  off  the  voice  part.  During  a  session,  the  users  go  through  the  sensemaking  process  of 
framing  the  context  of  discourse,  selecting  issues,  agreeing  on  a  position  of  discourse, 
and  presenting  arguments  to  validate  their  claims.  The  user  can  use  maps,  annotations, 
and  graphics  to  illustrate  facts  or  clarify  arguments  as  in  Exhibit  3. 


2  David  S.  Alberts  and  Richard  E.  Hayes  (2006).  Understanding  Command  and  Control.  CCRP  Publication 
Series  (http://www,dodccrp 


Exhibit  2.  S3  Screen  capture 


Exhibit  3.  S3  screen  with  information  search  (file  and  web  browsing) 

In  the  left  hand  side  of  the  figures  below  (Exhibit  4),  the  user  can  visualize  areas  of 
interests  and  likely  IED  routes.  The  risk  levels  are  classified  using  data  fusion  using 
coded  symbols.  On  the  right  hand  side,  the  user  can  visualize  situation  updates  by 
browsing  through  daily  events,  including  link  analysis  to  other  C2  centers  (or  cities),  and 
specific  areas  of  interest  that  were  targeted  (e.g.,  mosques).  The  last  screen  shot  in 
Exhibit  5  is  an  example  of  information  link  network  processed  through  likehood 
relationship  metric.. 


Exhibit  4.  Visualized  area  of  interest 


Exhibit  5.  Information  linkage  network 


EXPERIMENT 


Subjects: 

Six  experienced  military  commanders  participated  in  the  study.  The  participants 
consisted  of  four  Lt.  Colonels  and  two  Majors.  The  subject  population  had  one  female 
with  the  rank  of  Lt.  Colonel  in  the  Air  Force  and  also  the  commander  of  North  Carolina 
A&T  State  University  (NCA&Tj’s  ROTC  Air  Force  Regiment.  There  were  5  males:  4  of 
the  male  participants  are  retired  (3.  COL  and  1  Major).  The  retired  Major  is  on  a  reserve. 
The  female  officer  had  command  experience  with  Special  Air  Force  Command,  and  all 
the  male  participants  with  the  rank  of  Lt.  Colonel  had  experience  commanding  either 
field  artillery  or  infantry  battalions.  All  participants  had  a  combined  military  experience 
of  1 16  man  years. 

Apparatus: 

The  apparatus  consisted  of  S3  software,  a  personal  computer,  and  1 8”  TV  monitor 

Procedure: 

Each  officer  participated  as  team  of  two  “battle  staffs”  sequenced  in  a  permutation 
of  6  objects  taken  2  at  a  time  to  yield  15  trials  with  3  repetitions  per  trial.  Each  team 
received  each  of  the  cases  shown  in  Exhibit  1  in  a  random  order  that  precludes  learning 
effect.  Each  person  serves  in  a  different  team  during  a  trial.  The  study  took  five  days  of 
one  hour  per  session  per  team.  The  participant  was  told  to  use  the  S3  software  to 
communicate  their  sensemaking  thoughts  and  situation  analysis  with  each  other.  Each 
team  was  given  sufficient  time  as  needed  to  complete  the  task.  An  example  transcript  is 
shown  in  Exhibit  6.  Each  transaction  between  team  members  was  saved  to  S3  spread 
sheet  and  Excel  files  for  future  analysis.  At  the  end  of  a  scenario,  the  participants  were 
asked  to  subjectively  rate  the  effectiveness  of  S3  as  a  sensemaking  tool.  This  is  the 
analysis  presented  in  this  report.  The  sample  questionnaire  measures  sensemaking  (SM) 
capability,  situation  awareness  (SA),  and  situation  understanding  (SU).  A  sample  given 
below: 


How  would  you  rate  the  sensemaking  software  (S3)  on  the  following  (use  a  scale  of  1-7: 
1  =  absolutely  very  poor  and  7  =  absolutely  very  good  and  useful): 

1.  Sensemaking: 

a.  Information  presentation  allows  for  concept  mapping: 

b.  Promotes  contextual  reasoning  for  sharing  ideas  and  concepts: 

c.  Allow  users  to  see  different  interpretation  of  situation: 

d.  Promotes  retrospective  information  search: 

e.  Promotes  prospective  sensemaking  through  predictive  analysis: 

2.  Situation  awareness: 

f.  Can  see  common  operating  pictures  of  the  situation: 

g.  Allows  me  to  see  area  of  interest: 

h.  Allows  me  to  see  information  changes  over  time: 

i.  Encourages  team  information  sharing  and  dialogs: 

j.  Captures  how  our  team  view  the  same  situation  in  different  ways: 

3.  Situation  understanding: _ 


k.  I  can  explain  situation  based  on  my  experience  and  the  information 
displayed: 

l.  1  can  describe  the  situation  in  terms  of  expectations 

_ m.  I  can  visualize  the  courses  of  action  more  rapidly. _ 


Result  of  Experiment 

The  experimental  data  was  stratified  into  three  categories  of  cognitive  measures; 
namely,  sensemaking,  situation  awareness,  and  situation  understanding,  respectively. 
Basic  statics  for  each  cognitive  measure  was  summarized  in  terms  of  average  scores 
across  45  trials  (15  grouping  *  3  replicates).  Tables  2-4  show  the  results  for  minimal 
constructed  scenario  (MCS)  number  2  shown  in  Exhibit  1.  For  each  cognitive  measure,  a 
factor  loading  in  terms  of  percentage  variance  contribution  of  each  element  within  the 
measure  was  obtained  using  Statistical  Analysis  Software  (Dobson,  2003);  the  result  was 
then  used  to  obtain  a  weighted  score  of  each  measure  as  a  linear  relationship. 


Table  2.  Statistical  Analysis  for  Sensemaking  Score  (rating:  1=  min;  7=max) 


Elements 

Min  score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contribution 
(factor 
loading)3 

Allows  for 
concept  mapping 

2.6 

3.2 

5.1 

1.5 

0.12 

Contextual 
reasoning  for 
ideas  sharing 

1.1 

2.8 

4.7 

0.8 

0.1 

Interpretation  of 
situation 

2.4 

4.3 

6.5 

1.3 

0.32 

Retrospective 
information  search 

2.0 

4.5 

6.8 

0.67 

0.26 

Predictive  analysis 

1.9 

3.1 

4.3 

0.96 

0.2 

aThis  gives  the  weighted  linear  composite  score  for  sensemaking  of  3.84  and  a  standard 
deviation  of  1 .04  (linear  weighted  product  column  6  and  column  3  (for  average)  and 
column  5  (for  standard  deviation)). 


Table  3.  Statistical  Analysis  for  Situation  Awareness  Score  (rating:  1=  min;  7=max) 


Elements 

Min  score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contribution 
(factor 
loading)13 

Common 
operating  picture 

4.2 

6.6 

7.0 

0.6 

0.27 

See  area  of 
interest 

3.3 

4.8 

6.5 

0.91 

0.16 

See  information 
changes  over  time 

2.0 

3.5 

5.8 

1.2 

0.07 

Team  information 
sharing  and  dialog 

4.6 

6.8 

7.0 

0.2 

0.38 

Team  situation 

awareness 

3.1 

3.9 

5.4 

1.3 

0.12 

TT - - - - - - - - 

This  gives  the  weighted  linear  composite  score  for  sensemaking  of  5.85  and  a 
standard  deviation  of  0.46  (linear  weighted  product  column  6  and  column  3  (for 
average)  and  column  5  (for  standard  deviation)). 


Table  3.  Statistical  Analysis  for  Situation  Understanding  Score  (rating:  1=  min;  7=max) 


Elements 

Min  score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contribution 
(factor 
loading)0 

Explain  situation 

1.5 

3.2 

4.9 

1.6 

0.3 

Describe  situation 

2.0 

3.4 

4.7 

1.3 

0.22 

Visualize  courses 
of  action 

4.8 

6.1 

7.0 

0.08 

0.48 

cThis  gives  the  weighted  linear  composite  score  for  sensemaking  of  4.64  and  a  standard 
deviation  of  0.8  (linear  weighted  product  column  6  and  column  3  (for  average)  and 
column  5  (for  standard  deviation)). 


Data  was  analyzed  for  all  three  scenarios — simple,  complex,  and  chaos  problems, 
respectively.  Analysis  of  variance  (ANOVA)  was  used  to  investigate  any  interaction 
effect  between  problem  scenarios  and  the  three  cognitive  measures.  An  ANOVA  with  3 
by  3  randomized  block  design  was  used  to  evaluate  any  different  in  average  scores  across 
cognitive  levels  and  problem  types.  The  results  show  the  followings:  (a)  the  perception 
rating  of  S3  with  respect  to  the  cognitive  measures  was  highly  significant,  F  (2,  37  =2.4  < 
5.8  compute  value;  (b)  the  problem  scenarios  were  highly  significant,  F  (2,  37)  =  2.4< 
3.23;  and  there  was  noticeable  interaction  effect  between  the  cognitive  measures  and  the 
problem  types,  F  (4,  37)  =  2.09  <  1 1.3  computer  value.  All  tests  were  conducted  with  5% 
level  of  significant.  Figure  4  shows  the  weighted  average  scores  for  each  cognitive  score 
across  the  three  problem  types  (MCS1,  MCS2,  and  MCS3). 


□  MCS1 

□  MCS2 

□  MCS3 


Figure  3.  Weighted  mean  scores  for  cognitive  measures  by  problem  type. 


SUMMARY 


This  paper  has  described  an  on-going  effort  to  develop  operational  visualization 
concepts  and  their  technical  implementations  to  support  sensemaking  skills  based  on  the 
Army’s  FM3.0  doctrinal  information  on  “Visualization,  Detection,  and  Decide” 
requirements.  First,  visualization  is  our  attempt  to  allow  the  sensemakers  to  “see  the  same 
thing”  in  place  and  time  so  as  to  gain  real-time  situation  awareness.  Through 
visualization,  the  team  members  can  share  their  mental  models,  present  their  perspectives 
either  textually  or  graphically.  S3  also  satisfies  the  doctrinal  requirement  by  using  signs, 
symbols,  and  signals  to  invoke  informational  cues,  which  through  display,  allows  real¬ 
time  noticing  emerging  information  features  for  enhancement  of  decision  clues  to  the 
sensemaker.  Finally,  the  S3  software  provides  the  decision  maker  with  macro-level 
cognition  useful  for  building  situation  understanding  relevance  to  tasks,  doctrines,  and 
the  requirements  for  a  mission  or  goals.  We  have  started  the  validation  of  S3  using  Army 
cadets  from  Fort  Leavenworth.  The  results  show  the  followings:  (a)  the  perception  rating 
of  S3  with  respect  to  sensemaking  cognitive  measures  was  highly  significant;  b)  the 
problem  scenarios  (MCS)  were  highly  significant;  and  there  were  noticeable  interaction 
effects  between  the  cognitive  measures  and  the  problem  types.  Generally,  the  S3  software 
needs  further  improvement  in  representation  fidelity  of  problems  that  mimic  battle  field 
situations.  This  is  shown  by  the  poor  weighted  ratings  in  a  more  chaotic  scenario  MCS3. 
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APPENDIX 

Sample  text  transaction  between  two  intended  uses  of  S3 

Listening  to  port:  1007 

Connection  request  id  780  from  152.8.96.23 

Gwang  is  Logged  on. 

ntuen:  Major  Gwang,  we  have  a  problem 

Gwang:  What  is  the  problem...? 

ntuen:  Najaf  attacked.  100  killed,  new  group  claim  responsibility 
Gwang:  What  should  we  do....? 

ntuen:  Sensemaking,  major,  find  this  people,  their  name,  affiliation,  sponsor,  let’s  go  after  them 
Gwang:  Do  you  know  that  location? 

ntuen:  See  the  marked  area  of  interest  in  the  mark.  And  direction  of  their  movement 

Gwang:  I  am  by  Kuwait  border.  See  my  marking  with  rectangle. 

ntuen:  I  can  see  you  major,  get  your  men.  What  is  the  best  way  to  get  there? 

Gwang:  See  map  with  the  red  arrows.... 

ntuen:  Bad  choice,  IED  and  A1Q  group  are  everywhere 

Gwang:  I  look  for  alternative  way... 

ntuen:  I  am  checking  with  Alpha  3. 1  have  sent  the  map  with  your  routes  1  &  2  to  him 
Gwang:  How  long  can  I  wait....? 

ntuen:  Start  at  1400  hours.  That  is  4  hours  from  now  to  get  your  men 
Gwang:  What  is  the  probability  of  attack  in  Najaf? 
ntuen:  My  intel  and  record  up  till  3  days  ago  is  about  65% 

Gwang:  Can  you  tell  me  more  about  attack  statistics  on  IED,  Mortar,  VIED? 

ntuen:  OK,  Alpha  has  it  right  in  the  TOC:  IED  is  90%,  Mortar  is  daily  and  about  90%,  VIED  is  about  55% 
Gwang:  OK,  I  will  run  my  Sensemaking  program... 

ntuen:  Good,  get  your  COAs  in  order  and  be  prepared  to  encounter  trouble  on  the  way 

Gwang:  OK,  three  other  locations  marked  in  red  from  south  of  Turkey,  A1  Anbar  province,  and  Baghdad. 

ntuen:  Good  major,  I  will  notify  the  commanders  over  there.  So  now  we  have  to  involve  5  commands,  including 
mine,  right? 

Gwang:  Yes,  Sir. 

ntuen:  Ok,  get  back  to  me  at  1  hour  before  your  men  leave  to  Najaf 
Gwang:  Talk  to  you  then... 
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INTRODUCTION 


<\<=>e 


Z  WANT  TO  KNOW 
WHO'S  RESPONSIBLE 


1 .  What  is  relevance  in  what  they 
see? 

2.  Are  they  seeing  the  same  thing? 

3.  Do  they  have  the  same 
interpretation  in  context? 

4.  How  does  the  situation 
understanding  enable  action? 


X  GAVE  YOUR  ORPERS 
TO  CAPT.  SCABBARD 
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Sensemaking  Challenge 

T o  create  a  systematic, 
widespread  and  persistent 

Cognitive  Edge  for  the 

warfighter 


Effect 
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Sensemaking:  An  End-to-End  Approach 
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Adapted  from  "Understanding  Information  Age  Warfare"  (CCRP,  2001) 
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Search  is  the  mind’s  eye, 

But  sensemaking  is  the  mind’s  muscle. 


Stuart  Card 

PARC 

Collection  without  sense-making,  both 

automated  and  human, 

is  both  wasteful  and  falsely  reassuring 
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Robert  David  Steele,  CEO  of  OSS. Net, 
March  25,  2006 
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It  would  sure  be 
nice  if  we  had 
some  clear  idea 
what  it  was  we 
were  trying  to  do 
first 
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Adm.IVlJke  Boordal 
1938  1996 
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What  is  sensemaking? 


r 


DERIVING  MEANING  FROM  FRAGMENTARY  CUES- 
(DARPA’S  Information  Awareness  Project 


COLLECTING  “DOTS”  and  BRIDGING  MEANING  TO 
HUGE  VOLUME  OF  DATA— INQ-Tel  (Arlington-based 
company). 


A  SYSTEM  OF  ACTIONS,  SYMBOLS  AND  PROCESSES 
THAT  ENABLES  AN  ORGANIZATION  TO  TRANSFORM 
INFORMATION  INTO  VALUED  KNOWLEDGE  WHICH 
INTURN  INCREASES  ITS  LONG-RUN  ADAPTIVE 
CAPACITY  -  (Schandt,  1997;  pp.  8) 
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According  to  Franks,  battle  command  means 
seeing  what  is  now,  visualizing  the  future  state  or 
what  needs  to  be  done  to  accomplish  the  mission 
and  then  knowing  how  to  get  your  organization 
from  one  state  to  the  other  at  least  cost  against  a 
given  enemy  on  a  given  piece  of  terrain.  The 
primary  components  of  battle  command  that 

depend  directly  on  the  commander’s  intuition  are 

decision  making,  visualizing,  concept  formulation 

and  battlefield  awareness-selecting  the  critical 

time  and  place  to  act,  and  knowing  how  and  when 

to  make  adjustments  during  the  fight. 

Using  judgment  acquired  from  experience, 
training,  study,  and  creative  thinking, 
commanders  visualize  the  situation  and 
make  decisions.  In  unclear  situations, 
informed  intuition  may  help  commanders 
make  effective  decisions  by  bridging  gaps 
in  information.  Through  the  art  of 
command,  commanders  apply  their 
values,  attributes,  skills,  and  actions  to 
lead  and  motivate  their  soldiers  and 
units...  (FM  3.0) 
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Elements 
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Military  Doctrinal  Elements 


The  sheer  volume  of  information 
can  camouflage  the  critical 
information  we  need.  We’re  still 
working  on  our  ability  to  glean 
through  this  information  and  find 
the  necessary  information 
nuggets  that  will  aid  in  decision 
making  (MG.  Kamiya,  2007) 
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&  technology 
ndeavors 


Doctrines 


Mission  Command 
•CDR  intents,  Order 
•Guidance,  PIR 
•CCIR,  EEFI 
•Staff  initiatives 
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s — W - W - W - W - W 

Analytical 
Support:,  e.g 
OODA& 

DSS  for 
Planning 

▼  ▼  ▼  ▼  ▼ 

Visualize/Describe/Explain/Predict/Control 

Optimized  COA  K 

Running  estimates 
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Decision 

Support 

Systems 


Anticipate 


Influence 


DIME 


Affect  J  CDR  Decision 
Making  Points 


PMESII 


Measurements 


J  1  ;  1  Decision  Superiority 

Report/Review  Assessment 

Meter  (DESAM) 


Human: 
Adaptive,  insightful 
Collaborative, 
leadership 
Decision  quality 


Information: 
Relevancy,  timeliness 
Availability,  reliability \ 
Trust,  quality,  etc 


Technology: 
Resiliency,  robust, 
Reliable,  adaptable 
Adaptive, 


Lead 


/  -  D 
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-  Doctrine 

-  Principles  of  war 

-  Operational  themes 

-  Experience  and  judgment 


T 


Understand 


The  Problem 


Operational  Environment 
Enemy 


METT-TC 


\mi\rz3 


The  End  State  and 
the  Nature  and 
Design  of  the 
Operation 

Offense 
Defense 
Stability 
Civil  Support 


T 
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Describe 

Time,  Space, 
Resources, 
Purpose,  and 
Action 

Decisive  Operations 
Shaping  Operations 
Sustaining  Operations 


Direct 

Warfighting 

Functions 


Movement  and  Maneuver 

Intelligence 

Fires 

Sustainment 
Command  and  Control 
Protection 


•  Initial  commander’s  intent 

•  Planning  guidance 

•  Commander’s  critical 
information  requirements 

•  Essential  elements  of 
friendly  information 


Assess 
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Plans  and  orders 
Branches  and  sequels 
Preparation 
Execution 
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BATTLE  COMMAND 


The  Art  and  Science  of 


Battle  Command 


CDR/ Staff 
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Lead 

Principles  of  War, 

Tenets, 
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CDR  /Staff 

Describe 
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Factors  of  the 
Situation 
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Elements  of 
Operational 
Design 

•End  State  &  conditions 
•Center  of  Gravity 
•Decisive  Points 
•Lines  of  Operation 
•Culminating  Point 

•Operational  Reach  & 
Approach 

•Simultaneous  & 
Sequential  Ops 


Time,  Space, 
Resources, 
Purpose  & 
Action 

1  Decisive  Ops 
1  Shaping  Ops 
'  Sustaining  Ops 


•Linear  &  Nonlinear 
OPs 
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\JEnymty7  } 

•Planning 

guidance 

•Cdr’s  Intent 

Staff  Running 
Estimates 

Input  from 
other 

Commanders 

cdr/  staff 

Direct 

\ART/  Science k 


•Plans  & 
Orders 

•Preparation 

•Execution 


•Tempo 


Effects 


In  short,  we  need  to  develop  an  integrated  approach  for  the 
understanding  (framing)  and  visualizing,  describing,  directing, 
assessing,  and  reframing  of  unified  operations. 
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LandWarNet  provides  the  full  spectrum  of  connectivity  -  from 
the  deployed  Soldier  to  Home  Station  Operations  Centers, 
National/Strategic  Intel  Centers  and  Logistic  Support  & 
Sustainment  locations  -  encompassing  Joint,  Interagency,  and 
Multinational  capabilities. 
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THE  THEORY  FOR  DISPLAY  DESIGN  & 
VISUALIZATION 


1 .  Human  cognitive  processes 

2.  Display  theories 

3.  Visualization  modalities  &  techniques 

4.  Decision  performance 
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A  TETRAHEDAL  MODEL 
LINKING  VISUALIZATION, 
COGNITION,  DISPLAY, 
AND  SU 


Deliberate 

process, 

Mnemonic 

devices, 

enable 

first-level  SA. 

Provide 

sampled  data 

elements 


Mission,  commander  intents,  doctrines, 
focus,  effects,  intuition/tacit  knowledge, 
insights,  hindsight,  foresights 


**<£«*> 


High  level  SA  that  the  mind  cannot 
perceive,  organized  knowledge,  map 
actionable  knowledge  to  goals, 
causal  linkages,  understand  risks, 
provide  running  estimates,  etc. 


Envision  the 
Problem  space 
With  critical  features 


Focal  knowledge  sharing 
Critical  thinking 
Organized  knowledge 

isualiz^ 

Card,  et  al.  (1999): 

1.  Increase  cognitive  resources 

2.  Reducing  search 

3.  Enhance  pattern  recognition 

4.  Easy  perceptual  monitoring 

5.  Seeing  the  features  of  on¬ 
going  events 

6.  Provide  a  manipulable 
medium 


Cognition  and  Visualization  in  Situational  Understanding:  An 
Abstraction  Model 
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Cognitive  Enablers  and  Filters 


Visualization 


Perception 


Characterization  of  the 
adversary;  Envisioning; 
Mapping  actionable 
knowledge  to  effect 

Perspective  taking  & 
assumptions 
Frame  hypotheses 
Sensemaking 

Cognizing  decision 
elements,  cues,  clues, 
signs,  symbols,  and  signals 
&  enabling  neural  activities 

Environmentally  embodied; 
Perception  of  first-level  cues, 
followed  by  the  instantiation  of 
recognition-primed  activities; 


Information  portrayal;  Display  of 
tangible  information  elements  of  the 
situation  to  support  cued  cognition; 
Afford  a  nces 


Information  Display  Rendering 


How  Visualization  Enables  the  Commander’s  Cognitive  Process 
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Risk  Constants 
1.  Risk: 

RPGs  0.4 


2.  Comments : 

Shoulder-fired  Rocket  Propelled  Granades. 


Profile  -1/4 

1.  Events: 

Traffic  Jam 

05dul-06  09:30  -  11:00 

2.  Comments : 

Usually  there  is  traffic  jam  in  downtown  area 
around  this  time. 


Our  Current  S3 
(version  1 .0)  for 
Collaborative 
Sensemaking 
and  information 
fusion  capability 


Sample  MCS  (Minimally  Constructed  Stories) 


Case-1 

Captain  XX  just  receive  intelligence  reports  on  possible  attacks 
on  critical  infrastructure  including  kidnapping  key  political  and 
religious  leaders. 

Prepare  a  sensemaking  and  inform  the  commander  on  the 
available  courses  of  action  to  respond. 

Case-2 

General  Haltrack  is  informed  that  a  group  of  terrorists  has  been 
arrested  in  node  K  south  of  the  adversary  network.  Intel  showed 
that  they  have  spread  to  other  cells  in  the  network,  but  are  not 
identifiable. 

Use  a  sensemaking  model  to  analyze  the  situation.  Include 
information  on  the  terrorists  cell — who  they  are,  sponsors,  their 
strategies,  how  the  evolve,  their  motives,  etc.  Work  with  all 
intelligent  organizations. 
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PRELIMINARY  VALIDATION  RESULTS 


6  Experienced  military  officers: 

4  LTC,  2  Majors;  5  males,  1  female. 
2  from  ROTC;  2  from  BCBL; 

1  reserved  component;  1  retired. 
116  man  years  of  service. 

Minimal  Constructed  Scenarios 
Developed  from  Cynefin  Problem 
Typologies: 


MCS1 :  Knowable 
MCS2:  Complex 
MCS3:  Chaos 

SM  =  Sensemaking  perception 
SA:  Situation  awareness  perception 
SU:  Situation  understanding  perception 
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PRELIMINARY  VALIDATION  RESULTS:  Sensemaking  perception  score  (1-7:  1 
absolutely  not  useful  ;7  =  absolutely  very  useful) 


Elements 

Min  score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contribution 
(factor 
loading) 

Allows  for 
concept 
mapping 

2.6 

3.2 

5.1 

1.5 

0.12 

Contextual 
reasoning  for 
ideas  sharing 

1.1 

2.8 

4.7 

0.8 

0.1 

Interpretation  of 
situation 

2.4 

4.3 

6.5 

1.3 

0.32 

Retrospective 

information 

search 

2.0 

4.5 

6.8 

0.67 

0.26 

Predictive 

analysis 

1.9 

3.1 

4.3 

0.96 

0.2 
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PRELIMINARY  VALIDATION  RESULTS:  Situation  Awareness  perception 
score  (1-7:  1 -absolutely  not  useful  ;7  =  absolutely  very  useful) 


Elements 

Min  score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contribution 
(factor 
loading) 

Common 

operating 

picture 

4.2 

6.6 

7.0 

0.6 

0.27 

See  area  of 
interest 

3.3 

4.8 

6.5 

0.91 

0.16 

See  information 
changes  over 
time 

2.0 

3.5 

5.8 

1.2 

0.07 

Team 

information 
sharing  and 
dialog 

4.6 

6.8 

7.0 

0.2 

0.38 

Team  situation 

awareness 

3.1 

3.9 

5.4 

1.3 

0.12 
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PRELIMINARY  VALIDATION  RESULTS:  Situation  Understanding  perception 
score  (1-7:  1 -absolutely  not  useful  ;7  =  absolutely  very  useful) 


Elements 

Min 

score 

Average 

Max  score 

Standard 

deviation 

%  variance 
contributio 
n  (factor 
loading) 

Explain 

situation 

1.5 

3.2 

4.9 

1.6 

0.3 

Describe 

situation 

2.0 

3.4 

4.7 

1.3 

0.22 

Visualize 
courses  of 
action 

4.8 

6.1 

7.0 

0.08 

0.48 

4) 

r 

h 
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Summary  and  Results 


The  results  show  the  followings: 

(a)  the  perception  rating  of  S3  with  respect  to  the 
cognitive  measures  was  highly  significant,  F  (2,  37 
=2.4  <  5.8  computed  value; 


(b)  the  problem  scenarios  were  highly  significant, 
F  (2,  37)  =  2.4<  3.23; 


c 

a 

o 


(c)  there  was  noticeable  interaction  effect  between 
the  cognitive  measures  and  the  problem  types,  F 
(4,  37)  =  2.09  <  11.3 


01 

r 
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Research  Extension  to  SASO  Planning 
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1.0.  Mission  Alert 


1  -  Strongly  Agree 

4  -  Undecided 

7  -  Strongly  Disagree 

2  -  Agree 

5  -  Disagree  Somewhat 

3  -  Agree  Somewhat 

G  -  Disagree 

|  [Stabilize  Government! 

P  Provide  Safety  and  Security 
P  Defeat  Insurgency 
P  Provide  Self-Sustaining  Economy 

1.1.  Non-military  Tactical  Elements 

P  Gain  Trust 

P  Adapt  Attitude  and  Beliefs 
P  Know  Key  Players 
P  Minimum  Language  Skill 

1.2.  Measure  of  Performance  (MOP) 

1.  Casualties 

-  To  Blue  Force 

P  Death 

P  Wounded 

-  To  Civilians 

P  Death 

P  Wounded 

2.  Displacements 

P  Refugee  Camps  Within 
P  Evaluated  Outside 
P  Asylums 

3.  Stable  Government 

i”  Stabilize  Government 
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